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INTRODUCTION 



Common observation and research (Harris, 1961 Betel, 1968; Bond 
and Tinker, 1967) Indicate that many elementary grade children are asked 
to read books and materials that are too difficult for them. Although 
the condition is largely the result of the graded system which generates 
assumptions that all children learn all things at virtually the same time, 
it seems imperative that teachers place students in materials which are 
commensurate with the students' reading skills. 

Seemingly, the first teaching task must be one of determining the 
appropriateness of reading materials for the various students. To some 
extent, the standardized achievement tests which are frequently offered 
at least once a school year in most school systems, provide such informa- 
tion. However, as many teachers note, the results of such tests do not 
provide a reliable index of reading success in various mate, rials. The 
reasons for this are basically the following; 

1) Achievement tests are based on limited samples and 
cannot predict achievement accurately in specific 
materials which draw on varied concepts, sentence 
patterns, etc. 

2) Achievement tests are most reliable in the middle 
ranges of achievement and are consequently often 
very misleading in measuring the achievement of 
those in the lower ranges (Harris, 1962; Bond and 
Tinker, 1967). 

Because of the limited value of standardized tests for determining 
the suitability of given reading materials for given students, many 
reading authorities (Betts, 1946; Johnson and Kress, 1967; Harris, 1962; 
Barbe, 1961) suggest informal tests of the involved materials. That is, 
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the best test of reading skill resides in the student’s ability or 
inability to read the given material. Thus if a sixth grade teacher 
wishes to determine which students can profitably read the sixth 
grade geography text, the teacher must? 

1. Direct each student to read a specified portion of the 
text , 

2, Direct the student to demonstrate some degree of under- 
standing (whatever is deemed basic by the teacher). 

Such understanding is normally done by answering questions 
about the selection. 

Such testing in the materials is generally called informal 
reading inventory testing. In most instances the label is equated with 
the task of finding pupil’s reading levels by asking them to read a series 
of increasingly difficult selections (followed by comprehension ques- 
tions) . Students in the earlier stages of reading development read the 
various materials both orally and silently while higher level students 
normally read only silently before answering the questions. 

Although potentially valuable, informal reading inventory testing 
involves many qualitative decisions on the part of the teacher such as? 

Or al Reading 

’,/hat are oral reading errors? 

What are the maximum number of oral reading errors that can 
be permitted? 

How fluent should the oral reading be? 

How do you determine fluency? 

S ile nt Reading 

What is a reasonable amount of time to read the given selection? 

Comnveb ensrt.on 

What are the most important elements that the student should 
remember about the selection? 

To what extent are the questions relevant to the main elements 
of the selection? 

It must be evident that the quality of teacher judgments in inventory 
reading assessment is dependent upon very sophisticated judgments. In 
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fact, the judgments can be so sophisticated that certain reading 
theoreticians suggest that teachers may make completely inappropriate 
judgments if they use the prevailing error marking systems (Goodman, 

1967; Hunt, 1969 Powell, 1968). 

At this point the question may well be asked, * If teache rs cannot 
dgMgyercen t tes ts or thei r ow n observ ation s for determining 
Pi w ading mat erials fov_di jl^ren^chi Hr en, what, then 
£i? the y use?' The response to this question has been made in two very 
different ways. One prominent means has been the effort of several 
diagnostic reading test authors to develop tests that can more accurately 
predict the proper instructional level. Spache (1963), Botel (1968), and 
others have presented data to indicate that their special instruments 
will predict more accurately than achievement tests. Another prominent 
means has been seen in the ” cloze technique” procedure as developed by 
Bormuth (1967 a , 1968), in the cloze procedure, students are asked to 
restore omitted words (usually every fifth word) in a reading passage. On 
the basis of correct restorations, Bormuth (1967 a , 1968) and Coleman (1969) 

indicate that rather accurate determinations of comprehension can be 
made. 

Because the tests of Botel and Spache (as any other standardized 
instruments) suffer from the same limitations as achievement tests, it 
appears that their power in determining the appropriateness of reading 
material is somewhat limited. Devoid of such restraints and geared to the 
exact material, the cloze test procedure offers a most valuable means of 
determining the readability of any selection for any student. Thus, the 
study was designed to seek further information about the 



predic tive value of the cloze te st in the e lcmentar y grades . Such 
determ ina tion was to be ma d e by compari ng the results o f individual 
pupil 1 s sc ores on c lo ze tes t s as well as o ther types of reading 



measures , 
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PROBZiEM 



Students are, as previously Indicated, often asked to read 
materials which are beyond their reading capabilities. In order to 
make a valid judgment of whether or not a student can read given 
materials, it seems imperative to test the student directly in the 
materials. Taylor (1957), Bormuth (1967 a), Jenkinson (1957), and 
Coleman (1962) have revealed research support to indicate that cloze 
tests (involving the restoration of omitted words) can be used as 
valid and reliable predictors of reading comprehension. The current 
research was designed to further study some additional aspects of the 
cloze tests’ ability to determine reading comprehension. 

In order to determine how well cloze tests predict readability it 
was essential to determine readability by some other means. Thus, the 
current study sought to make comparisons of the cloze test with the 
Botel Reading Inventory and the reading subtest (Comprehension) of the 
Metropolitan Achievement Test. 

Botel (1968) reported that his reading inventory correlated very 
highly with the placements made by a group of study teachers using care- 
fully prescribed informal reading inventory techniques. (The correlations 
ranged from as high as .95 in Grade 2 to the low of .73 in Grade 6). In 
making similar comparisons between various standardized achievement test 
scores and the teachers’ findings of instructional levels, he found that 
the standardized tests faired considerably poorer than his instrument. 

From his research, he concluded that the Botel Inventory was more closely 
related to the criterion than the standardized silent reading tests 




used. 
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Because the Botel correlations were consistently higher in the 
intermediate grades than those of the standardized achievement tests, 
the decision was made to use the Botel Reading Inventory 9 Form A. 

(Word Opposites) as the criterion of instructional level. Against this 
criterion score for each child itfould be placed his instructional score 
on the cloze test and his metropolitan Achievement Test (Reading 
Comprehension) score. Specifically, the study was addressed to the 
following hypotheses: 

Hypothesis 1 

There is no significant relationship between the 
cloze test instructional levels and those of the 
Botel Beading Inventory Form A (Word Opposites) 
in Grades 4, 5, and 6. 

Hypo thes is 2 

There is no significant relationship between the 
Metropolitan Achievement Test (Reading Comprehension) 
scores and those of the Botel Reading Inventory 
Form A (Word Opposites) in Grades 4, 5, and 6. 

Hypothesis 3 

There is no significant relationship between the 
cloze test Instructional levels and the Metropolitan 
Achievement Test (Reading Comprehension) levels in 
Grades 4, 5, and 6. 

Hypothesis 4 

There is no significant difference between the 
relationship of the cloze test instructional levels 
and the criterion and those of the Metropolitan 
Achievement Test (Reading Comprehension) and the 
criterion in Grades 4, 5, and 6. 




7 



BACKG ROUND LITERATURE 

A study of the project title "A Comparative Study of the Validity 
of the Cloze Test and Metropolitan Achievement Test (Reading Comprehension 
Subtest) for Making Judgments of Ins tructional Levels' suggests the 
presence of a valid criterion of instructional level. Actually, such 
validity is in question because the validity in this study is based upon 
research by Botel (1968), wherein he found high, positive correlations 
between his reading test and teacher judgments of Instructional levels 
in a certain school. Conceivably? what was reported as validity may have 
been only reliability between the two sets of scores. At issue are the 
basic determinants of the ’'Instructional level,' In this review the 
discussion will center upon the concept of the so-called "instructional 
level 15 or "optimum level of difficulty" and the means relative to the 
determination of such. 

The Ins tr uctional Level 

Beldin (1969), traces the beginnings of the concept of an instructional 
level (or level of difficulty which appears optimum for instruction) well 
beyond Killgallon (1942), and Betts (1946), and found the concept in the 
early writings of Gray (1925), Thorndike (1934), and others. 

VJhat the optimum determinants of reading difficulty happens to be 
remains a subject of great delate. On the one side can be found Individuals 
of the Betts persuasion who appear to accept the following criteria: 
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Levels of Reading Difficulty 



Word Recognition 



Comprehension 



Independent 



99% 



90% 



Instructional 



95% 



75% 



Frustrational 



90% or less 



50% or less 



On the opposite side we find individuals such as Powell (1968) , Hunt (1969), 
Spache (1969), and others who find the Killgallon-Betts criteria to be 
arbitrarily fashioned and not commensurate with reality. Both Spache and 
Powell indicate studies that have demonstrated that comprehension (at the 
Betts 1 level) can be obtained with significantly lower word recognition 
skill than outlined by the criteria. Powell found that first and second 
graders could on the average comprehend at the 70% level with 85% word 
recognition. 

If word recognition criteria are dropped and the focus is placed 
entirely upon the comprehension factor there remains disagreement as to the 
minimum level of acceptable comprehension. Whereas the Killgallon-Betts 
criterion was 75%, Spache feels that a comprehension of 60% is acceptable. 
Frequent modifications of the initial I.R.I. concepts frequently imply a 
70% criterion because of the pattern of using a ten question format. 

When the multitude of variables which surround the informal reading 
inventory concepts are taken into consideration, it becomes very difficult 
to determine what optimum functioning really is. So difficult indeed is 
the process, that McCracken (1964), deemed it useful to add a rate dimension 
that would further identify the nature of the instructional task, i.e. 
indicate a plodding reader who might be both accurate in word recognition 
and comprehension. 
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Conceivably, much research remains before we can fix specific percentage 
criteria for determining optimum degrees of instruction. Such research must 
certainly describe the effects of varying types of instruction upon varying 
types of pupils (emotional makeup, etc.) using varying kinds of materials. 
Achi evement Tests and Instructional Levels 

As indicated in the introduction, achievement tests have been criticized 
as useful measures of instructional reading levels. The problem of using 
standardized reading tests for making judgments about instructional levels is 
very clearly illustrated by McDonald (as cited by Emans, Urbas, and Dumett, 
1966) in discussing a research project in a high school that administered 
different reading tests to the same students with differing results. Butel 
(1968) , revealed evidence which indicated that achievement test placements 
were very far off when low group children were concerned. His results revealed 
discrepancies to the extent that more children were improperly placed by the 
standardized tests than were properly placed. Reports of others (Harris, 

1961, Bond and Tinker, 1967), reveal similar pictures, especially with regard 
to children who deviate from the mean reading levels tested by the various 
tes ts . 

It should be pointed out that achievement tests are basically instruments 
designed for broad achievement assessment and not the assessment of an 
individual’s ability to read a given text on a given day in a given situation. 
Thus, it is not highly reasonable to anticipate book placement on the basis 
of such instruments. 

Cloze Tests and Instructional Level 

Wilson L. Taylor (1953) introduced the term "cloze procedure" in 
1953 and initiated considerable research relative to the value of closure 
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tasks as predictors of leading comprehension. 

Basic to the procedure is the idea of closure wherein the reader 
must use the surrounding context in order to restore omitted words. 
Comprehension of the total unit and its available parts (including the 
emerging cloze write-ins) is essential to the task. As the task has 
been described by Taylor , Bormuth (1968), and others, cloze tests have 
generally involved the following administration and scoring protocols: 

Administration 

1. Every nth word (usually every 5th word, Bormuth 
(1963), is omitted and in its place a blank of 
sufficient length allowed for the student to write 
in the ansvrer. 

2. The student is instructed to write only one word 
in each blank and to try to fill in every blank. 

3. Guessing is encouraged. 

4. Students are advised that spellings will not be 
counted as errors. 

Scoring 

1. In most instances, the exact word must be restored 
(Bormuth, 1968). 

2. Misspellings are counted as correct when the response 
is deemed correct in a meaning sense. 

The validity of the cloze test as a measure of readability and comprehen- 
sion has been most interesting because of (1) the ways in which it has been 
accomplished and (2) the almost universal finding of high correlations 
between cloze and other prediction instruments. 

Initially Taylor (1953) compared cloze score rankings of passages of 
varying difficulty with readability rankings of the same passages by two 
common readability formulas (Dale-Chall, 1948, Flesch, 1949) . The passages 
were similarly rank ordered by each technique. Superiority for the cloze 
procedure was demonstrated when very difficult passages could be more readily 
assessed as such by the cloze procedure. 
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Another way in which the cloze test has been validated has been by 
comparing results between cloze tests and standardized tests of reading 
achievement. Potter (1968) reveals in Table 1 the following correlations of 
a number of such studies % 



Table 1 

Correla tions Between Clo ze Readability Tests and 
Standard ized T ests of Reading Achievement 



Study 


Subjects 


Tests Correlations 


Jenkinson (1957) 


High School 


Cooperative Reading C2 


.78 






Vocabulary 






Level of Comprehension 


.73 


Rankin (1957) 


College 


Diagnostic Survey 


.29 




Story Comprehension 






Vocabulary 


.68 






Paragraph 


.60 


Fletcher (1959) 


College 


Cooperative Reading C2 


.63 






Vocabulary 






Level of Comprehension 


.55 






Speed of Comprehension 
Dvorak~Van Wagenen 


.57 






Rate of Comprehension 


.59 


Hafner (1963) 


College 


Michigan Vocabulary Profile 


.56 


Ruddell (1963) 


Elementary 


Stanford Achievement 


-.74 


(5 cloze tests) 




Paragraph Meaning .61* 


Weaver and Kingston 


College 


Davis Reading .25- 


-.51 


(1963, 2 cloze tests) 






Green (1964) 


College 


Diagnostic Reading Survey 


.51 






Total Comprehension 
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Bormuth (1967 a) demonstrated a relationship between cloze and multiple- 
choice scores of subjects and further determined that a 43 percent cloze 
score was the equivalent of a 75 percent multiple -choice test score (with 
corrections made for guessing). Thus, the link between cloze testing and 
the concepts of Instructional level determination was made (using the Betts- 
ICillgallon comprehension criterion of 75%). It appears from the research, 
that Bormuth has equated cloze performance with the criterion (regardless 
of the accuracy of the criterion) . 

As suggested earlier, it is interesting to note the consistency with 
which the cloze test correlates with other measures of comprehension. The 
only notable exception appears to be a study by Kingston and Weaver (1963) 
wherein the researchers measured comparably low correlations of .25 and .51 
between cloze test scores and scores of college students on the Davis Reading 

Test. 

Although most of the studies of cloze have taken place from the fourth 
grade and up, two studies are reported for the lower grades. Gallant's (1965) 
data suggested that the technique was appropriate for first, second and third 
grade children. Changes such as the insertion of three words for each blank 
for choice purposes (a multiple-choice task) and the lengthening of certain 
sentences to obtain higher Spache Readability levels raise questions about the 
study. Deutsch (1964) utilized a cloze task via an auditory means (by asking 
students to fill in the word as the voice model paused) and found that cloze 
tests were highly related to I.Q. 

To date, the background literature suggests that the concept of an 
instructional or 1 optimum level of reading instruction 11 is a concept that 
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Is widely discussed, but which does not possess a necessarily common set 
of characteristics. 

Research relative to the comparative effectiveness of (a) standardized 
achievement tests and (b) Cloze tests. Indicates that the latter practice 
has greater predictive value. Thus, this study sought to further explore 
the comparative value by comparing Metropolitan and Cloze test scores. 
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RESEARCH PROCEDURES 



Sample Selection 

Prior experience with cloze testing of pupils below fourth grade 
reading skill had indicated to the researcher (although not to others, 
i,e», Jenkinsou, 1957) that the nature of the task was rather difficult. 
Tills experience suggested that the study might best be conducted with 
pupils reading 4t'n grade level and above. 

Arrangements were made to test pupils in the Brentwood Elementary 
School in Austin, Texas, who were reading at their respective grade 
levels (fourth, fifth, sixth) in the judgments of the teachers involved. 
In essence, this meant that fifty students were selected from each class 
of approximately one hundred and fifty students. 

The initial samples of fifty students were picked on the basis of 
their proximity to grade level expectation (as based on teachers’ 
judgments). Inevitably, intact reading groups were in the sample. 

Because of the unique planning arrangements, the final sample 
didn’t look quite as anticipated in that actual book placements were 
under those of the standardized achievement tests. For example, of the 
50 fourth graders, only 17 were reading a fourth grade level text. The 
balance were reading third grade level texts. Even more striking was the 
situation in the sixth grade where only eleven were reading a sixth grade 
material at the time of testing. 

Test Preparation 

In order to develop a cloze test that would be a representative 
measurement of each grade level the following criteria were established. 



o 
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1. Each selection used would have to meet Bormuth's 
specifications of lenght (250 words or 50 cloze 
items as a minimum) . 

2. Each test should be contained within a single page 
so as to facilitate close scrutiny by the student. 

3. Each selection should be representative of the 
difficulty of its purported level as determined 
by the Dale-Chall or Spache Readability Formulas 
(from book other than reader). 

A. Lines should be typed and double spaced so as to 
permit the student to easily manage the visual 
difficulties that might occur. Blanks were ten 
spaces in length so as to permit a reasonable space 
for pupil write-in answers . 

Information pertinent to the samples chosen is presented in the 
table below (Table 2). 

Table 2 

Cloze Samples 9 Their Sources , and Their 
Readability Levels as Determined by 
The Spache or Dale-Cliall Readability 
Formulas 



Grade Cloze Sample Title Publisher Re com. Level Readability 



2 


David's Silver 






2. A 


(Spache) 




Dollar 










3 


The Valentine Box 




3rd 


3.3 


(Spache) 


A 


The Brave Little 






A. 5 


(Dale-Chall) 




Tailor 










5 


Station in Space 






5. A 


(Dale-Chall) 


6 


The Squeak of 


Ginn 


6 th 


6.1 


(Dale-Chall) 




Leather 










7 


The River of the 


Ginn 


7 th 


7.1 


(Dale-Chall) 



Wolves 



After the cloze tests were prepared, testing sets for the three grade 
levels were made up. Each packet contained the following: 

Bote 1 Wo rd Opposites Test. 

Cloze Test . One Grade Below Presumed Grade Level. 

Cloze Test. On Presumed Grade Level. 




Cloze Test. 



One Grade Above Presumed Grade Level. 
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Thus, after completing the Botel test, each student xvrould begin working 
on a cloze test which was presumably one grade level below his current reading 
level. He was then supposed to progress through the other tests. Students 
who did not pass the lowest tests or who passed the highest tests were 
given further tests to determine their reading levels. 

Test Administration 



Arrangements were made for the selected sample group in each grade 
to meet in a large space (4th graders in a given room, 5th graders in 
another room, etc.). The following instructions were given to the 
examiners % 

1* ASK EACH CHILD TO WRI TE HIS /HER NAME ON THE TEST. Check 
to see that each child has written" hlsTher name on the 
first page of the test. 

2. EXPL AIN THAT T HERE ARE TWO KINDS OF TESTS IN THE PACKET. 

A. READ THE DIRECTIONS FOR THE FIRST'tYPE OF TEST (PICK A 
WORD IN EACH LINE WHICH MEANS THE OPPOSITE OF THE 
NUMBERED WORD) . READ THE EXAMPLE AND BE SURE THAT 
EVERYBODY UNDERSTANDS THE TASK. PAGE THROUGH THE FIRST 
FIVE PAGES AND SHOW THEM THAT THESE PAGES COMPRISE THE 
FIRST TYPE OF TEST. 

B. EXPLAIN THAT THE SECOND TYPE OF TEST IS A '’FILL IN THE 
BLANK” TYPE OF TEST AND THAT THEY ARE TO INSERT THE 
WORDS THAT SEEM TO BE NEEDED IN ORDER FOR THE SELECTION 
TO MAKE SENSE. SHOW THEM THAT THE LAST THREE PAGES OF 
THE TEST PACKET ARE TESTS OF THIS TYPE. FURTHER INDICATE: 

1. It is well to read through the whole ”Fill in the 
Blank” page first before writing in words. 

2. Spelling errors will not be counted, 

3. They may erase and change words if they like at any 
time . 

4. After they have finished each page they should read 
through it again to see if they are satisfied with 
their answers. 

5. If they simply can’t think of an answer for a space 
to go past it and come back to it later. 

STRESS THAT THE TASK WILL BE DIFFICULT AND THAT THEY SHOULD TRY NOT TO BE 
TOO DISCOURAGED IF THEY DON’T KNOW WHAT ANSWERS TO PUT IN. FURTHER EXPLAIN 
THAT WE HAVE SOME OF THE SAME PROBLEMS THAT THEY HAVE WHEN WE TAKE TESTS 
LIKE THIS. 
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Notes 

1. Circulate to determine whether the students understand the test tasks. 

2, Try to allow as much time as each individual needs to complete the 
tasks . 

3 • If this ap pears to be too much fo r on e s itting , pie ase make provisio ns 

to stop at som e p oint and c o mplete t he oth er tasks at anothe r sitting .' 

In the fourth and fifth grade testing sessions , the author was present 

and gave the initial instructions. He was not present in the sixth grade 

examination . 

Time for the examination varied rather sharply, with a variance of 
45 to 88 minutes in the fourth and fifth grades. 

It was noted by the examiner that the cloze test task in the fourth 
and fifth grades appeared quite difficult to the children. Many suggested 
that it was : too hard.” 

Sta tis tical Procedures 

Comparisons of the power of the two tests with regard to making 

instructional level decisions were made by (1) correlation and (2) graphic 

matching, using the Botel word opposites test (Form A) scores as the 
criterion scores. 

The various reading tests (Metropolitan, cloze, Botel) were correlated 
in paired fashion with the Pearson Product-moment correlation technique 
(Bruning & Kintz, 1968). Initially, the cloze results in the three grades 
were correlated with the criterion results (as obtained by the Botel). 

Next, the Metropolitan Achievement Test (Reading Comprehension) results in 
the same grades were correlated with the criterion results (as obtained by 
the Botel). Then, the results of the cloze and Metropolitan tests were 
correlated . 

The relationships of the cloze and Metropolitan test results to the 
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criterion were subsequently measured by a 15 1 1 ' test (Bruning and Klntz, 
1968 ) . 

For a determination of the relative matching of the cloze and 
Metropolitan tests with the criterion , a chart was developed wherein the 
placement agreements and disagreements (underplacement and overplacement) 
could be registered. 
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research findings 



It was the intent of the study to determine the validity of the cloze 
and Metropolitan tests for making judgments of instructional levels by 

testing the following hypothesis: 

1. There is no significant relationship between the cloze test 
instructional levels and those of the Botel Reading Inventory 
Form A (Word Opposites) in Grades 4,, 5 9 and 6 . 

2. There is no significant relationship between the Metropolitan 
Achievement Test (Reading Comprehension) scores and those of 
the Botel Reading Inventory Form A (Word Opposites) in Grades 
4 3 5, and 6 . 

3. There is no significant relationship between the cloze test 
instructional levels and the Metropolitan Achievement Test 
(Reading Comprehension) levels in Grades 4, 5, and 6 , 

4. There is no significant difference between the relationship 
of the cloze test instructional levels and the criterion and 
those of the Metropolitan Achievement Test (Reading Comprehen- 
sion) and the criterion in Grades 4, 5, and 6 . 

Each of these sets of findings is treated in the following material. 

Hypothesis 1 , There is no significant relationship between the cloze test 
Instructional levels and those of the Botel Reading Inventory Form A (Word 
Opposites) in Grades 4, 5, and 6 . 

This hypothesis was not rejected by the data from the correlation of 
the cloze and Botel Scores in each of the three grades. Results of the 
correlations indicated very low but positive correlations as follows: 

4th Grade - r. ,11 
5th Grade - r. .17 
6 th Grade - r. ,18 

Hypothesis 2. There is no significant relationship between the Metropolitan^ 
Achievement Test (Reading Comprehension) scores and those of the Botel Reading 
Inventory Form A (Word Opposites) in Grades 4, 5, and 6 , 

This hypothesis was rejected in the fourth and sixth grades but not in 
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the fifth grade. Results are as follows: 

4th Grade - r. .49 
5th Grade - r. .21 
6th Grade - r. .49* 

*s i gni f i can t at .05 

The failure to reject the hypothesis at the fourth and sixth grade levels 
indicates a fairly high correlation between these two standardized instruments 
at these levels. Seemingly, it would be anticipated that significant, positive 
correlations would exist at all three levels. 

Hypothesis 3 . There is no significant relationship between the cloze test 
instructional levels and the Metropolitan Achievement Test (Reading Compre- 
hension) levels in Grades 4, 5, and 6. 

This hypothesis was not rejected in the 4th and 5th grades. However, in 
the 6th grade there was a negative correlation (-.57) which was significant 
at the .05 level of significance. Seemingly 9 part of the difficulty seems to 
be accounted for by the difficulty of the seventh grade cloze test. Not a 
single sixth grader passed the seventh grade cloze selection, even though 
many scored much higher than grade level on the Metropolitan (and the Botel 
as well) . 

It is curious to note that the 7th grade selection "The River of the 
Wolves" checked out as 7.1 on the Dale— Chall Readability Formula. After the 
testing had been completed the Fry Readability Formula came to the attention 
of the author and the selection was tested with this instrument. The Fry 
formula revealed a difficulty of 5th grade. 

Hypothesis 4, There is no significant difference between the relationship 
of the cloze' test instructional levels and the criterion and those of the 
Metropolitan Achievement Test (Reading Comprehension) and the criterion in 
Grades 4 , 5, and 6. 



21 



This hypothesis was tested statistically by a "t"* test which revealed 
a significant relationship at the .05 level in the fourth grade (Table 3) . 
Thus, the hypothesis was rejected at this grade level. The hypothesis was not 
rejected in Grades 5 and 6. 

Table 3 

Comparison of r's Between (1) Metropo- 
litan Achievement Test (Reading Comprehen- 
sion) and (2) Cloze Tests and CRITERION 



Grade 


Metropolitan 


Cloze 


t 


4 


,49 


.11 


2.08 


5 


.17 


.17 


.05 


6 


CO 

H 

« 


.18 


.93 



* "Test for Difference Between Dependent Correlations" from Computational 
Handbook of Statistics by Janes L. Bruning and B. L. Kintz, Glenview, Illinois; 
Scott, Foresman, 1968, p. 193. 

Differences in the relationships of the two tests to the criterion were 
illustrated by comparison with the criterion (Table 4) . 

In the 5th and 7th grade facility levels as determined by the criterion 
test (Botel) , the Metropolitan Achievement Test placed more students on 
criterion level than did the cloze test. In the 6th grade, there was little 
difference with only one more child being placed correctly via the cloze. 

Underplacement appeared to be the result of most of the cloze test 
results. With the exception of fourteen students overplaced one year in the 
fifth grade and one overplaced two years in the fourth, some 98 students were 
underplaced from one to four years . 

Whereas underplacement appeared to be the result of most cloze test 
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placement, overplacement appeared to be the effect of the Metropolitan 
Achievement Test in the 5th and 6th grades „ In the 7th grade both cloze 
and Metropolitan tests appeared to overplace. 
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CONCLUSIONS 



The very 1cm correlations between the cloze score instructional levels 
and the instructional levels of the Botel Reading Inventory Form A (Word 
Opposites) seems rather strange in light of the Bormuth (1967 a), and Botel 
(1968), studies which indicated the strength of each instrument in assessing 
instructional levels. Seemingly, the results suggest one or more of the 
following possibilities: 

- The Botel Reading Inventory is not a valid measure of 
reading comprehension difficulty, 

- The cloze test technique (or the use of a single cloze 
test for assessment) is not a valid measure of reading 
comprehension difficulty. 

- The cloze test difficulty rankings by the readability 
formulas were not accurate. 

It should be pointed out that two tests are very different with regard 
to their means of assessing comprehension. The Botel (Word Opposites) Test 
determines comprehension on the basis of the student’s ability to read a word 
and select an opposite word from a group of alternative words. The cloze 
test requires the reader to establish clozure on a certain percentage of 
omitted words in a selection of at least 250 words. Seemingly, one might 
be able to perform the Botel task on the basis of individual word attack 
skill plus an understanding of key vocabulary. In the cloze task, the larger 
concerns of syntax and lexical meaning come into play. Thus, it seems possible 
that a student might be able to perform the individual vocabulary task without 
being able to do the latter successfully. Tlius, it is highly conceivable that 
the cloze test might be the more valid predictor of the ability of a pupil to 
read a given selection. Of course, the acceptance of such a generalization 
must rest heavily upon the validity of the research of Bormuth (1968), and 
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others who have demonstrated cloze validity by comparisons with other comprehen- 
sion checks, notably multiple choice. 

It was anticipated that high, positive correlations would exist between 
the Botel Word Opposites Test and the Metropolitan Reading Comprehension Test 
because of the fact that the standardization process, through which both have 
been placed, demands such. Thus, the significant, positive correlations in 
the fourth and sixth grades were anticipated. The low, non-significant 
correlations at the fifth grade was not anticipated. In studying the 
comparative results of the three grade groups in terms of their scores the 
following information was found with regard to the relationship of Metropo- 
litan and Botel scores: 

Table 5 

Comparability of Metropolitan and Botel 
Grade Placements in Grades 4, 5, and 6 



Grade 


No. taking 
Both Tests 


Same Score/ 
Both Tests 


Higher Score/ 
Botel 


Higher Score/ 
Metropolitan 


4th 




19 


12 


13 


5th 




23 


15 


8 


6th 




17 


5 
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While the magnitudes of difference which were reflected in the correlations 
are not evident in Table 5, it is apparent that the variance is rather 
marked between the three grades. Beginning with the fourth where there are 
about an equal number of higher scorers in each test, we see the Botel 
reflecting much higher scores in the 5th and the reverse occurring in the 
6th grade. 
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Explanations of the wide differences between the 5th and 6th grades 
are difficult. Possibly, the tests administered (either) to one or the other 
of the groups were not properly administered. Conceivably, one grade might 
have encountered a different difficulty level in test taking. All 
possibilities are speculative. 

The failure to obtain significant correlations between the cloze and 
Metropolitan tests (Hypothesis 3) in the 4th and 5th grades was not unexpected. 
However, the significant, negative correlation of -.57 at the 6th grade was 
surprising. As indicated the result seemed to be in large part, test 
difficulty on the part of the cloze test because no sixth graders passed the 
seventh grade cloze test, even though many scored higher on both Hotel and 
Metropolitan tests. Seemingly, the 7th grade selection "The River of the 
Wolves” should be submitted to a similar group of sixth graders as a complete 
story followed by a set of ten questions for comprehension testing. If the 
story appeared too difficult for most or all of such a group as based on 
a criterion of seven answers correct, the readability of the selection as 
determined by the Dale-Chall and Fry formulas would be further questioned. 

If however, comprehension proved obtainable, the researcher would have to 
question either (a) the validity of the cloze method for such selections or 
(b) the testing procedures under which the examinations were given. 
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IMPLICATIONS FOR FURTHER STUDY 



Comparisons of cloze test results with the results of global compre- 
hension measures as involved in such tests as the Botel Reading Inventory 
and Metropolitan Achievement Test (Reading Comprehension) do not appear 
to be the most important kinds of comparisons. Rather, comparisons should 
seek to further validate the cloze comprehension criteria by comparing 
cloze results with other measures of passage understanding such as multiple 
choice questions, open response questions, tell-back techniques, etc. 

While much of this research has been done, it seems that further validation 



is needed. 
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